Retieving a Web page

Latest update: June 2015

This tutorial demonstates how to get a web page from the internet using iSDIO commands.

Overivew

The iSDIO specification Wireless LAN addendum contains functions to send and receive data using HTTP (HyperText Transfer Protocol).

By using these functions, the FlashAir will perform any HTTP or TCP/IP (Transmission Control Protocol/Internet Protocol) processing instead of the Arduino.

This means we can quickly and easily handle basic communication functions without using RAM on the Arduino.

Now, lets retrieve a web page!

Types of HTTP commands

There are several different versions of the HTTP commands, for different protocols and ways to pass data.

Command ID	Name	Protocol	Request Data Type	Request Data From
21h	SendHTTPMessageByRegister	HTTP	Text	Memory
22h	SendHTTPFileByRegister	HTTP	File	Memory
23h	SendHTTPSSLMessageByRegister	HTTP over SSL	Text	Memory
24h	SendHTTPSSLFileByRegister	HTTP over SSL	File	Memory
25h	SendHTTPMessageByFile	HTTP	Text	File
26h	SendHTTPFileByFile	HTTP	File	File
27h	SendHTTPSSLMessageByFile	HTTP over SSL	Text	File
28h	SendHTTPSSLFileByFile	HTTP over SSL	File	File

The name of every function starts with Send, however they are also used to receive data. This is because in HTTP a client initiates communication by sending a request, and the server sends back a response.

A "get" request only requests for a response from a server, whereas a "put" request will send data with the request.

For more information about HTTP, please read Wikipedia .

Please note that the maximum size of response data that the FlashAir can receive is 2048 bytes, including the 24-byte iSDIO header.

SendHTTPSSLMessageByRegister

To start, we're going to download the front page of the FlashAir Developer's website.

Sending a request

FlashAir Developers uses encrypted communication by sending HTTP over SSL (Secure Socket Layer, HTTPS). We'll use the GET command to request data. GET only requires a simple request header with a few lines of text.

Command ID 23h, SendHTTPSSLMessageByRegister, meets the requirements above.

arduino_tutorial_6.ino (partially extracted)

boolean iSDIO_http(uint32_t sequenceId) {
  Serial.print(F("\nhttp command: \n"));
  memset(buffer, 0, 512);
  uint8_t* p = buffer;
  p = put_command_header(p, 1, 0);
  p = put_command_info_header(p, 0x23, sequenceId, 2);
  p = put_str_arg(p, "flashair-developers.com");  // Argument #1.
  p = put_str_arg(p,                              // Argument #2. 
      "GET /en/ HTTP/1.1\r\n"
      "Host: flashair-developers.com\r\n"
      "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36\r\n"
      "\r\n");
  put_command_header(buffer, 1, (p - buffer));
  printHex(buffer, (p - buffer));
  return card.writeExtDataPort(1, 1, 0x000, buffer) ? true : false;
}

Line 7:
Specify the domain name of a web server. An IP address is also acceptable.
Line 9:
GET command. Written in the format: GET <Path> <Protocol and version><line break>. HTTP/1.1 should work fine for the protocol/version. Note that line breaks must be \r\n in HTTP.
Line 10:
Speicify a host name.
Line 11:
Speicify a user agent. FlashAir Developers will warns if a web browser is not supported. In this example, we'll pretend to be a Chrome browser by specifying a Chrome user agent string. This way the web server will respond properly.
Line 12:
We need an empty line at the end of the request.

Receiving a response

Wait for command completion, then read the Response Data. Wait for command completion and reading response data are described in Starting in AP mode: Confirm command processing status.

In this example, the response data should only be text data - so lets just print it out!

iSDIO Command Response Data (SD Specifications Part E7 Wireless LAN Simplified Addendum Version 1.10)

Reading data from the Response Register Port needs to be done in 512-byte blocks, in case the response is large. (512 bytes is the maximum size available in the iSDIO Command Response Data.) This means that the N-th read of iSDIO Commmand Response Data returns N-th 512-byte block of a response.

arduino_tutorial_6.ino (partially extracted)

boolean iSDIO_httpResponse() {
  // Read header and data.
  if (!card.readExtDataPort(1, 1, 0x200, buffer)) {
    return false;
  }
  uint32_t totalSize = get_u32(buffer + 20);
  uint32_t availableSize = totalSize > 488 ? 488 : totalSize;
  uint32_t pos = 24;
  for (;;) {
    for (uint32_t i = 0; i < availableSize; ++i) {
      Serial.print((char)buffer[pos + i]);
    }
    totalSize -= availableSize;

    // Have we read all data?
    if (totalSize == 0) break;

    // Read next data.
    if (!card.readExtDataPort(1, 1, 0x200, buffer)) {
      return false;
    }
    availableSize = totalSize > 512 ? 512 : totalSize;
    pos = 0;
  }
  return true;
}

Lines 3-8:
Reads the first part of the response, and saves the information for later use.
Lines 9-24:
Shows response data on the screen. It will continue to read and display data in 512 byte chunks if the total size is more than 512 bytes.

Main program

Add a control routine to the main program.

arduino_tutorial_06.ino (partially extracted)

void loop() {
    .. (snip) ..
  Serial.print(F("\n5. HTTP Get"));
    .. (snip) ..
    case 5 :
      if (iSDIO_http(nextSequenceId) &&
          iSDIO_waitResponse(nextSequenceId)) {
          iSDIO_httpResponse();
        Serial.println(F("\nSuccess."));
      } else {
        Serial.print(F("\nFailed or waiting. errorCode="));
        Serial.println(card.errorCode(), HEX);
      }
      nextSequenceId++;
      break;
    .. (snip) ..
  }
}

Execution Results

Finally, let's execute the program!

Note: The following example requires a wireless LAN with an Internet connection.

Note: Please execute the Disconnect command if your FlashAir is already connected to any devices.

Initializing SD card...OK
Waiting response 
  Process Succeeded
... (snip) ...
0. Show status
1. Disconnect
2. Establish
3. Connect
4. Scan
5. HTTP Get

Command? (next sequence id = 1)

Run the Connect command to connect your FlashAir to a wireless LAN with an Internet connection. In this example, we're using iPhone Internet Sharing.

Input 3 in the box on the top of the serial terminal, and press Enter. After that, input the SSID and Network key. (Remember to terminate them with a ; (semicolon) !!)

SSID? iPhone
Network Key? 12345678

Connect command: 

00: 01010000300000000000000000000200
01: 0200000002000000060000006950686F
02: 6E650000080000003132333435363738

Waiting response 
  Command Processing.............
  Process Succeeded

Success.

Check the status message to make sure we have an IP address. ... (snip) ... [0550h] IP Address: 172.20.10.4 ... (snip) ...

Next, lets try an HTTP request. Input 5 in the box on the top of the serial terminal, and press Enter.

Command? (next sequence id = 2)

http command: 

00: 01010000E80000000000000000002300
01: 030000000200000017000000666C6173
02: 686169722D646576656C6F706572732E
03: 636F6D00AF000000474554202F656E2F
04: 20485454502F312E310D0A486F73743A
05: 20666C6173686169722D646576656C6F
06: 706572732E636F6D0D0A557365722D41
07: 67656E743A204D6F7A696C6C612F352E
08: 30202857696E646F7773204E5420362E
09: 333B20574F57363429204170706C6557
0A: 65624B69742F3533372E333620284B48
0B: 544D4C2C206C696B65204765636B6F29
0C: 204368726F6D652F33362E302E313938
0D: 352E313235205361666172692F353337
0E: 2E33360D0A0D0A00

Waiting response 
  Command Processing...
  Process Succeeded
HTTP/1.1 200 OK
Date: Tue, 21 Oct 2014 06:25:10 GMT
Server: Apache/2.2.15 (CentOS)
X-Powered-By: PHP/5.3.3
Set-Cookie: FASSID=c637h0hs62jvc9055f5krvpfc4; path=/
Cache-Control: max-age=0, no-cache, no-store, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: FASSID=o4365cm2e7nqf4afflk727flr3; path=/
Set-Cookie: FASSID=u5h3v3tf0ik7kn2jv3os2aakc4; path=/
X-Mod-Pagespeed: 1.8.31.4-4056
Vary: Accept-Encoding,User-Agent
Content-Length: 25710
Connection: close
Content-Type: text/html; charset=UTF-8

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7" lang="en"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8" lang="en"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9" lang="en"> <![endif]-->
<!--[if gt IE 8] -->
<html lang="en">
<!-- <![endif] -->


<head>

<meta charset="utf-8"/>
<!-- %meta{:content => "IE=edge,chrome=1", "http-equiv" => "X-UA-Compatible"} -->

<!--Google Apps verification-->

<title>FlashAir Developers - Home</title>
<meta content="width=device-width, initial-scale=1.0" name="viewport"/>

<!--[if lt IE 9]>
    <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
<link href="/images/flashair_favicon.ico" rel="shortcut icon"/>
<link href="/images/apple-touch-icon-114x114-precomposed.png" rel="apple-touch-icon-precomposed" sizes="114x114"/>
<link href="/images/apple-touch-icon-72x72-precomposed.png" rel="apple-touch-icon-precomposed" sizes="72x72"/>
<link href="/images/apple-touch-icon-57x57-precomposed.png" rel="apple-touch-icon-precomposed"/>
<link href="/css/bootstrap.css" media="screen" rel="stylesheet" type="text/css"/>
<link href="/css/responsive.css" media="screen" rel="stylesheet" type="text/css"/>
<link href="/css/font-awesome.css" media="screen" rel="stylesheet" type="text/css"/>
<link href="/css/theme.css" media="screen" rel="stylesheet" type="text/css"/>
<link href="/css/fonts.css" media="screen" rel="stylesheet" type="text/css"/>
<link href="/css/prett
Success.
... (snip) ...

The received HTTP data will be between HTTP/1.1 200 OK and Success.. The data may be truncated because of the data size limitation.

Sample code

arduino_tutorial_06.zip (24KB)

All sample code on this page is licensed under BSD 2-Clause License.

Retieving a Web page

Overivew

Types of HTTP commands

SendHTTPSSLMessageByRegister

Sending a request

arduino_tutorial_6.ino (partially extracted)

Receiving a response

iSDIO Command Response Data (SD Specifications Part E7 Wireless LAN Simplified Addendum Version 1.10)

arduino_tutorial_6.ino (partially extracted)

Main program

arduino_tutorial_06.ino (partially extracted)

Execution Results

Sample code

FlashAir™ Developers

Twitter

Sitemap